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Abstract. The rapid growth of online education courses, especially Massive Open 
Online Courses (MOOCs), has called attention to the issue of student retention 
and low overall completion rates (Gaebel, 2013). The impact of different modes of 
delivery on retention has also received attention with a blended learning mode being 
deemed most effective in retaining students (Harker & Koutsantoni, 2005). Fischer 
(2007) underscores the need for computer-based tracking data to expose students’ 
progress online. The study presented here is part of a larger study on student 
retention in online language learning courses and draws on tracking data from over 
43,000 learners on ‘Icelandic Online’ (IOL), an open online course in Icelandic as 
a second language. Previous findings reveal that completion rates on IOL are low 
and vary by mode of delivery, revealing regular attrition patterns across all modes 
of delivery. This paper demonstrates the importance of re-examining parameters for 
measuring retention and to correlate student retention to views on course content 
and platform. Further surveys based on the tracking data will elicit more in-depth 
knowledge about student engagements. 
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1. Introduction 


IOL offers seven courses in Icelandic as a second language for adults. The courses, 
developed at the University of Iceland, have been offered since 2004. These self- 
instructed online courses attract thousands of learners worldwide; providing the 
opportunity to learn Icelandic, unhindered by limits of geography. Two of the IOL 
courses (IOL 1 and 2) are offered in three delivery modes; as (1) an open and free, 
self-directed course, (2) a distance course, and (3) a blended course. 
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The proliferation of MOOCs has called attention to the issue of student retention 
and concern about low overall completion rates (Gaebel, 2013). It has been pointed 
out that retention is commonly measured without accounting for student intentions 
(Koller, Ng, & Chen, 2013) and that no student is obliged to complete or engage in 
a MOOC (Sokolik, 2014). 

Researchers have suggested that traditional approaches to measuring retention may 
not apply to MOOCs. The field of learning analytics have a significant role to 
play in providing valuable teaching and learning insights, and researchers (Long 
& Siemens, 2011) have called for more studies using this methodology on why 
students decide to leave online courses. 

This paper presents an analysis of tracking data on student retention collected over 
eight years on 43,000 users of IOL. It also introduces a follow-up survey-based 
study, currently underway, which seeks to provide a more in-depth understanding 
of why students decide to stay on or leave online courses. 


2. Method 

This is a three phased, mixed method study. A tracking system is an integral part 
of IOL and monitors learners’ engagement as they move through the course(s). 
The main objective of the first phase of the study is to analyse the large amount 
of tracking data by (1) mapping out overall retention rates for all the seven IOL 
courses, (2) exploring the effect of different modes on student retention, and (3) 
investigating what the overall engagement patterns on IOL suggest about the nature 
of retention. 

The next two survey-based phases will gather evidence from self-reports from the 
same students on their use of the IOL programme as measured by the tracking data, 
with a view to investigate why online learners decide to leave, as well as stay, to 
the end of the programme. The focus is on learners on IOL 2 who have covered 40- 
100% of the course in the three different modes of delivery: blended, distance, and 
open. The survey includes a large number of factors associated with instructional 
features of the programme and different learning environments. The goal is to 
investigate (1) whether certain course content factors on IOL serve to encourage 
or discourage retention, (2) why the blended mode is more effective than other 
modes in retaining students, and (3) whether learners’ intentions and motivations 
for taking the course are important factors in retention. Regression analysis is used 
to determine which variables affect retention. 
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3. Results 


3.1. Overall retention on IOL 

and the effect of different modes of delivery 

The data on overall retention on IOL reveal that overall completion rates are low 
across the seven courses: from 2.4% to 18.2%, depending on courses (Fridriksdottir, 
2017). Furthermore, when studying the effect of different modes of delivery on 
retention in two of the courses (IOL 1 and 2), the blended learning mode is the most 
effective in keeping learners in both courses. Table 1 shows the effects of the three 
modes on retention on IOL 2. 


Table 1. Course completion in different modes of delivery in IOL 2 



Open Self- 
directed Course 

Distance Course 

Blended Course 

N of beginners in a course 

3,462 

62 

281 

N of completers in a course 

152 

3 

40 

Completion rates 

4.4% 

4.8% 

14.2% 


The blended learning mode, with 14.2% completion rates, is more effective in 
retaining students than both the distance mode (4.8% completion rates), and the 
open self-directed mode (4.4% completion rates). The completion rates on the 
three different modes of delivery on IOL 1 revealed similar findings. 

3.2. Overall engagement patterns on IOL 

The retention data were broken down to reveal when non-completers dropped out. 
Two of the courses (IOL 1 and 2) were examined specifically for this purpose. 
Figure 1 shows that the tracking data revealed a regular drop-out pattern across 
all modes of delivery among the learners who did not remain to the end of IOL 2, 
showing concentrations of drop-outs at specific junctures in the course and large 
drop-out rates initially, irrespective of modality. The drop-out pattern in IOL 1 
revealed similar findings. 

The drop-out patterns exposed in Figure 1 raise further questions on the overall 
online engagement patterns of non-completers. IOL 1 and 2 were first investigated 
by using different parameters for coverage of a course. Table 2 reveals quite a 
different pattern when the parameters for coverage of course content are re¬ 
evaluated; completion rates are about (1) two times higher in all the three modes 
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on IOL 2 when the parameters for course completions are modified from 100% 
coverage of course content to 90%, (2) three to five times higher in all modes when 
adjusted to 75% coverage, and (3) four to eight times higher in all modes when 
changed to 50% coverage of course content. The results show similar engagement 
patterns when data from IOL 1 are examined. The findings presented in Table 2 
reveal immense impact on retention when the parameters used to measure retention 
are adjusted. 


Figure 1. Drop-out patterns across the three different modes of delivery in IOL 2 



Table 2. Retention in view of different coverage of course content in different 
modes of delivery in IOL 2 



Content 

coverage 

100% 

Content 

coverage 

90% 

Content 

coverage 

75% 

Content 

coverage 

50% 

Blended Course 

Completion rates 

14.2% 

29.2% 

40.6% 

53.4% 

Distance Course 

Completion rates 

4.8% 

6.5% 

25.8% 

40.3% 

Open Course 

Completion rates 

4.4% 

7.3% 

14.2% 

20.4% 


These findings raise further questions on student engagement patterns on all 
the IOL courses. They were analysed further for a more nuanced picture of the 
overall engagement patterns. Figure 2 exposes the patterns on all seven IOL 
courses showing that the majority of learners on IOL complete less than 50% of 
a course’s content. 
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Figure 2. Overall engagement patterns in all seven courses of IOL in terms of 
different parameters for coverage of course content 


Overall engagement patterns in all seven courses of IOL in terms of different parameters 
for coverage of course content 
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4. Discussion 

The results of the tracking data analysis support findings of previous studies on 
overall low completion rates in online courses (Gaebel, 2013) and that blended 
learning is most effective in retaining students (Harker & Koutsantoni, 2005). 
These findings reveal regular attrition patterns across all modes of delivery, with 
sharp drop-outs initially and concentrations of drop-outs at certain junctures in the 
courses, and that once parameters for measuring retention are adjusted, a different 
picture of retention is revealed. The findings give reason to ask what it really means 
to ‘complete a course’. 


5. Conclusions 

The analysis of IOL’s tracking data revealed interesting engagement patterns across 
courses and calls for a re-evaluation of how retention is measured in MOOCs. 
The next two phases of this study will gather self-reports from the same tracked 
students on their use of IOL to gain a deeper understanding of why online learners 
decide to leave, or complete, a programme. 
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